fix: add Gemini -latest aliases so a new API key can actually run the pipeline - #379
Open
ayo0la wants to merge 1 commit into
Open
fix: add Gemini -latest aliases so a new API key can actually run the pipeline#379ayo0la wants to merge 1 commit into
ayo0la wants to merge 1 commit into
Conversation
… pins No Gemini model in providers.json completes a request on a newly created API key. Probing each shipped model against the same endpoint OpenAICompatibleProvider uses: gemini-2.0-flash 429 quota gemini-2.0-flash-lite 429 quota gemini-2.5-flash 404 "no longer available to new users" gemini-2.5-flash-lite 404 "no longer available to new users" gemini-2.5-pro 429 quota The 404s are the hard failure: those two versions are retired at the model level for new keys, so no quota or billing change recovers them. The 429s are reported as observed rather than as proven config bugs, since project quota can't be fully ruled out. This bites new contributors specifically, because .env.example and the README both steered them to exactly those retired pins — a first run with a fresh key failed with an error that reads like a bad API key rather than stale config. Adds the gemini-flash-latest and gemini-flash-lite-latest aliases, which track the current version instead of going stale, and points the docs at them. The existing dated pins are left in place: they still resolve for keys that already had access, and removing them would break those users for no benefit. Verified end to end with DEFAULT_MODEL=gemini-flash-lite-latest on resume/sample.pdf --role software_engineering_intern: extraction, evaluation, and scoring all complete and return well-formed output. Both new aliases return 200 on a fresh key; the retired pins are unchanged. Fixes interviewstreet#378
Author
|
Still reproduces on
Worth noting the 404 set has moved since I first filed this in July: back then One note for triage, since there is overlap: #367 also touches this, but is narrower and now partly stale.
Happy to close this in favour of #367 if you would rather take that one, though that would still leave the docs pointing at models that do not resolve. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
No Gemini model in
providers.jsoncompletes a request on a newly created API key. Two of the five fail with a hard 404 model-availability error, and they are exactly the two.env.examplerecommends.Probed each shipped model against the same endpoint
OpenAICompatibleProvideruses (POST https://generativelanguage.googleapis.com/v1beta/openai/chat/completions), with a key created today:gemini-2.0-flashgemini-2.0-flash-litegemini-2.5-flash... is no longer available to new usersgemini-2.5-flash-lite... is no longer available to new usersgemini-2.5-proThe 404s are the hard failure — those versions are retired at the model level for new keys, so no quota or billing change recovers them. I am deliberately not claiming the 429s as proven config bugs: I had spent quota on that key earlier the same day and cannot fully rule out ordinary project quota. They are listed as observed.
Why this bites new contributors specifically
.env.examplerecommended"gemini-2.5-pro"and"gemini-2.5-flash", andREADME.mdusedgemini-2.5-proandgemini-2.0-flashas its worked examples. Following the README verbatim with a fresh key failed on the first run, with an error that reads like a bad API key rather than stale config.Fix
Adds the two
-latestaliases, which track the current version rather than going stale, and points.env.exampleandREADME.mdat them:gemini-flash-latestgemini-flash-lite-latestThe existing dated pins are left in place on purpose. They still resolve for keys that already had access, and removing them would break those users for no benefit. The docs simply stop recommending them, and the README now notes why aliases are preferable.
This is the part that stops the bug recurring — the same retirement will hit any
gemini-3.xpin in a few months.Testing
Both new aliases return 200 on a freshly created key against the repo's own endpoint; the five existing pins are unchanged.
providers.jsonvalidates as JSON, andconfig.provider_for()resolves both new entries to the Gemini provider with the expectedtemperature/top_p.Full end-to-end run with
DEFAULT_MODEL=gemini-flash-lite-latest:Section extraction, evaluation, and scoring all complete and return well-formed output —
0.0/100with coherent per-category evidence, which is the correct result for the bundled non-engineer sample resume.Reported in #378. Fixes #378